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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
05/02/2008 has been entered. 

Response to Arguments 

2. Applicant's arguments, see Remarks, filed 05/02/2008, with respect to the 
rejection(s) of claim(s) 13,14, and 28 under 35 U.S.C. 1 02(b) and claims(s) 1-12,1 5- 
26, and 29-30 under 35 U.S.C. 103(a) have been fully considered and are persuasive. 
Therefore, the rejection has been withdrawn. However, upon further consideration, a 
new ground(s) of rejection is made in view of Binnig et al. US 6792418 B1 (hereinafter 
Binnig). Though the previous reference of Chino teaches discourse analysis theory, 
Examiner has incorporated a new reference that is directed more specifically to the 
present invention. Binnig teaches the analysis of input strings that can be handled 
separately by a person or machine after processing occurs, wherein a grammatical 
parser is used to provide multiple frameworks for analysis based on various linguistic 
theories to output to a human/machine. 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-30 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational 
Speech?" (hereinafter Shriberg) in view of Binnig et al. US 6792418 B1 (hereinafter 
Binnig). 

Re claims 1,15, 29, and 30, Shriberg teaches a method of synthesizing speech 
(Page 5) using discourse function level prosodic features (Pages 14-18) comprising the 
steps of: 

determining discourse functions in the input text the discourse functions being 
determined based on a mapping between basic discourse constituents of the 
determined theory of discourse analysis and a plurality of discourse functions (Pages 8- 
13); 

determining a model of discourse function level prosodic features (Pages 14-18); 

determining adjusted synthesized speech output (Page 5) based on the 
discourse functions and the model of discourse function level prosodic features (Pages 
14-18) 
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However, Shriberg fails to teach determining a theory of discourse analysis from 
a plurality of theories of discourse analysis; 
determining input text; 

Binnig teaches a grammatical parser to create a syntactical structure of the input 
string. This syntactical structure exhibits the functional relation of the input string's 
elements. The grammatical theories of Slot Grammar, Lexical Functional Grammar, 
General Phrase Structure Grammar, or Discourse Representation Theory, all 
extensions of the ground-breaking work of Noam Chomsky (Syntactic Structures, 1959) 
provide possible frameworks for this step. It creates the input (formal) network 18 from 
the functional structure (Binnig Col. 14 lines 27-37)) 

Additionally, Binnig teaches a semantic preprocessor 17 creates a first guess of 
the resulting augmented input network by assigning semantical units (such as objects, 
attributes, connection objects, etc.) to the segments, individual words, individual 
keywords, or even semantical units (if the input string 12 contains any) of the input 
string 12, and connecting these semantical units with connection objects 3 (such as 
horizontal exchange, horizontal relation, scaling exchange, scaling relation, attribute 
connections 4, role connections 5, etc.) as the corresponding segments, individual 
words, individual keywords, or semantical units that they represent are deemed to be 
connected in the input string 12. Then the semantic processor 19 reads out the 
possible subsets from the knowledge database 1 1 or alternatively from the index 
database 13 that are deemed to be associated with the various semantical units of the 
above first guess of an augmented input network (which corresponds to the input string 
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12 and is generated by the semantical preprocessor 17). It performs a matching of the 
semantical units of the above guess with semantical units from the knowledge database 
1 1 or alternatively from the index database 13 through classification rules (such as 
inheritance, implementation, and overwriting rules). The resulting network is then called 
the augmented input network (Col. 16 line 61 - Col. 17 line 19 & Fig. 2c). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Shriberg to incorporate the determination 
of input text and the determination of a theory of discourse analysis from a plurality of 
theories of discourse analysis as taught by Binnig to allow for the augmentation of a 
knowledge database, wherein a system can learn the relationship between various 
terms in the input text and learn to semantically process input text with improved results 
at each iteration prior to outputting refined text (Col. 16 line 61 - Col. 17 line 19 & Fig. 
2c). 

Re claims 2 and 16, Shriberg teaches the method of claim 1, wherein the 
discourse functions are determined based on the determined theory of discourse 
analysis (Pages 8-13). 

Re claims 3 and 17, Shriberg fails to teach the method of claim 2, in which the 
theory of discourse analysis is at least one of: the Linguistic Discourse Model, the 
Unified Linguistic Discourse Model, Rhetorical Structures Theory, Discourse Structure 
Theory and Structured Discourse Representation Theory; 
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Re claims 4 and 18, Shriberg teaches the method of claim 1 , wherein the output 
information (Pages 4-5, Why Use Prosody?) is at least one of text information and 
application output information (Pages 8-13). 

Re claims 5 and 1 9, Shriberg teaches the method of claim 1 , wherein 
determining the adjusted synthesized speech output (Pages 4-5, Why Use Prosody?) 
further comprises the steps of: 

determining a synthesized speech output (Pages 4-5, Why Use Prosody?) based 
on the input text; 

determining discourse function level prosodic feature adjustments (Pages 14-18); 

determining the adjusted synthesized speech output based on the synthesized 
speech output (Pages 8-13) and the discourse level prosodic feature adjustments 
(Pages 14-18). 

However, Shriberg fails to teach determining input text; 

Binnig teaches a grammatical parser to create a syntactical structure of the input 
string. This syntactical structure exhibits the functional relation of the input string's 
elements. The grammatical theories of Slot Grammar, Lexical Functional Grammar, 
General Phrase Structure Grammar, or Discourse Representation Theory, all 
extensions of the ground-breaking work of Noam Chomsky (Syntactic Structures, 1959) 
provide possible frameworks for this step. It creates the input (formal) network 18 from 
the functional structure (Binnig Col. 14 lines 27-37)) 
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Additionally, Binnig teaches a semantic preprocessor 17 creates a first guess of 
the resulting augmented input network by assigning semantical units (such as objects, 
attributes, connection objects, etc.) to the segments, individual words, individual 
keywords, or even semantical units (if the input string 12 contains any) of the input 
string 12, and connecting these semantical units with connection objects 3 (such as 
horizontal exchange, horizontal relation, scaling exchange, scaling relation, attribute 
connections 4, role connections 5, etc.) as the corresponding segments, individual 
words, individual keywords, or semantical units that they represent are deemed to be 
connected in the input string 12. Then the semantic processor 19 reads out the 
possible subsets from the knowledge database 1 1 or alternatively from the index 
database 13 that are deemed to be associated with the various semantical units of the 
above first guess of an augmented input network (which corresponds to the input string 
12 and is generated by the semantical preprocessor 17). It performs a matching of the 
semantical units of the above guess with semantical units from the knowledge database 
1 1 or alternatively from the index database 13 through classification rules (such as 
inheritance, implementation, and overwriting rules). The resulting network is then called 
the augmented input network (Col. 16 line 61 - Col. 17 line 19 & Fig. 2c). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Shriberg to incorporate the determination 
of input text as taught by Binnig to allow for the augmentation of a knowledge database, 
wherein a system can learn the relationship between various terms in the input text and 
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learn to semantically process input text with improved results at each iteration prior to 
outputting refined text (Col. 16 line 61 - Col. 17 line 19 & Fig. 2c). 

Re claims 6 and 20, Shriberg teaches the method system of claim 1 , wherein the 
model of discourse function level prosodic features (Pages 14-18) is a predictive model 
of discourse functions (Page 19). 

Re claims 7 and 21 , Shriberg teaches the method of claim 6, in which the 
predictive models are determined based on at least one of: machine learning and rules 
(Page 19). 

Re claims 8 and 22, Shriberg teaches the method of claim 1 , in which the 
prosodic features occur in at least one of a location: preceding, within and following the 
associated discourse function (Page 14). 

Re claims 9 and 23, Shriberg teaches the method of claim 1 , in which the 
prosodic features are encoded within a prosodic feature vector. 

Re claims 10 and 24, Shriberg teaches the method of claim 9, in which the 
prosodic feature vector is a multimodal feature vector (Pages 14-18 & Table 10). 
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Re claims 1 1 and 25, Shriberg teaches the method of claim 1 , in which the 
discourse functions include an intra-sentential discourse function (Page 8 & Table 1). 

Re claims 1 2 and 26, Shriberg teaches the method of claim 1 , in which the 
discourse functions include an inter-sentential discourse function (Page 8 & Table 1). 

Re claim 13, Shriberg teaches a method of synthesizing speech using discourse 
function level prosodic features comprising the steps of: 

determining discourse functions in the input text based on a contextually aware 
theory of discourse analysis using a mapping between basic discourse constituents of 
the contextually aware theory of discourse analysis and a plurality of discourse 
functions (Pages 8-13); 

determining a model of discourse function level prosodic features (Pages 14-18); 

determining adjusted synthesized speech output (Pages 4-5, Why Use 
Prosody?) based on the discourse functions and the model of discourse function level 
prosodic features (Pages 14-18). 

However, Shriberg fails to teach determining input text; 

Binnig teaches a grammatical parser to create a syntactical structure of the input 
string. This syntactical structure exhibits the functional relation of the input string's 
elements. The grammatical theories of Slot Grammar, Lexical Functional Grammar, 
General Phrase Structure Grammar, or Discourse Representation Theory, all 
extensions of the ground-breaking work of Noam Chomsky (Syntactic Structures, 1959) 



Application/Control Number: 10/785,199 Page 10 

Art Unit: 2626 

provide possible frameworks for this step. It creates the input (formal) network 18 from 
the functional structure (Binnig Col. 14 lines 27-37)) 

Additionally, Binnig teaches a semantic preprocessor 17 creates a first guess of 
the resulting augmented input network by assigning semantical units (such as objects, 
attributes, connection objects, etc.) to the segments, individual words, individual 
keywords, or even semantical units (if the input string 12 contains any) of the input 
string 12, and connecting these semantical units with connection objects 3 (such as 
horizontal exchange, horizontal relation, scaling exchange, scaling relation, attribute 
connections 4, role connections 5, etc.) as the corresponding segments, individual 
words, individual keywords, or semantical units that they represent are deemed to be 
connected in the input string 12. Then the semantic processor 19 reads out the 
possible subsets from the knowledge database 1 1 or alternatively from the index 
database 13 that are deemed to be associated with the various semantical units of the 
above first guess of an augmented input network (which corresponds to the input string 
12 and is generated by the semantical preprocessor 17). It performs a matching of the 
semantical units of the above guess with semantical units from the knowledge database 
1 1 or alternatively from the index database 13 through classification rules (such as 
inheritance, implementation, and overwriting rules). The resulting network is then called 
the augmented input network (Col. 16 line 61 - Col. 17 line 19 & Fig. 2c). 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time of the invention to modify the system of Shriberg to incorporate the determination 
of input text as taught by Binnig to allow for the augmentation of a knowledge database, 
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wherein a system can learn the relationship between various terms in the input text and 
learn to semantically process input text with improved results at each iteration prior to 
outputting refined text (Col. 16 line 61 - Col. 17 line 19 & Fig. 2c). 

Re claims 14 and 28, Shriberg teaches the method of claim 13, in which the 
context is at least one of: semantic, pragmatic, and syntactic context (pages 4-5) 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. US 681 0378 B2, US 5751 906 A, US 58901 1 7 A, US 
20020078091 A1 , US 200201 42277 A1 , US 2004004451 9 A 1 , US 20050042592 A1 , 
US 20070073533 A1 , US 5732395 A, and Jurafsky et al., "Automatic Detection of 
Discourse Structure for Speech Recognition Understanding", Automatic Speech 
Recognition and Understanding, 1997. Proceedings., 1997 IEEE Workshop.. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michael C. Colucci whose telephone number is (571)- 
270-1847. The examiner can normally be reached on 9:30 am - 6:00 pm, Monday- 
Friday. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Richemond Dorvil can be reached on (571)-272-7602. The fax phone 
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number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Michael C Colucci/ 
Examiner, Art Unit 2626 
Patent Examiner 
AU 2626 
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Michael.Coiucci@uspto.gov 
/Richemond Dorvil/ 

Supervisory Patent Examiner, Art Unit 2626 



